In [1]:

    

cd distributed-louvain-modularity/example


    

    




/home/vagrant/distributed-louvain-modularity/example

In [2]:

    

./run_example.sh


    

    




clear out old example dir..Deleted /tmp/louvain-giraph-example
loading example data to /tmp/louvain-giraph-examplerunning the full louvain pipeline

running full louvain on /tmp/louvain-giraph-example/input
saving output to: /tmp/louvain-giraph-example/output

Removing ouput dirDeleted /tmp/louvain-giraph-example/output

running giraph phase /tmp/louvain-giraph-example/input -> /tmp/louvain-giraph-example/output/giraph_1
rm: `/tmp/louvain-giraph-example/output/giraph_1': No such file or directory14/01/26 03:24:41 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/01/26 03:24:41 INFO utils.ConfigurationUtils: Setting custom argument [giraph.vertex.input.dir] to [/tmp/louvain-giraph-example/input] in GiraphConfiguration
14/01/26 03:24:41 INFO utils.ConfigurationUtils: Setting custom argument [ mapreduce.task.timeout] to [10800000] in GiraphConfiguration
14/01/26 03:24:41 INFO utils.ConfigurationUtils: Setting custom argument [actual.Q.aggregators] to [1] in GiraphConfiguration
14/01/26 03:24:41 INFO utils.ConfigurationUtils: Setting custom argument [minimum.progress] to [2000] in GiraphConfiguration
14/01/26 03:24:41 INFO utils.ConfigurationUtils: Setting custom argument [progress.tries] to [1] in GiraphConfiguration14/01/26 03:24:41 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)14/01/26 03:24:41 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.14/01/26 03:24:43 INFO mapred.JobClient: Running job: job_201401260304_000214/01/26 03:24:44 INFO mapred.JobClient:  map 0% reduce 0%14/01/26 03:24:59 INFO mapred.JobClient:  map 50% reduce 0%14/01/26 03:25:00 INFO mapred.JobClient:  map 100% reduce 0%14/01/26 03:25:02 INFO mapred.JobClient: Job complete: job_201401260304_0002
14/01/26 03:25:02 INFO mapred.JobClient: Counters: 34
14/01/26 03:25:02 INFO mapred.JobClient:   File System Counters
14/01/26 03:25:02 INFO mapred.JobClient:     FILE: Number of bytes read=0
14/01/26 03:25:02 INFO mapred.JobClient:     FILE: Number of bytes written=410326
14/01/26 03:25:02 INFO mapred.JobClient:     FILE: Number of read operations=0
14/01/26 03:25:02 INFO mapred.JobClient:     FILE: Number of large read operations=0
14/01/26 03:25:02 INFO mapred.JobClient:     FILE: Number of write operations=0
14/01/26 03:25:02 INFO mapred.JobClient:     HDFS: Number of bytes read=432
14/01/26 03:25:02 INFO mapred.JobClient:     HDFS: Number of bytes written=134
14/01/26 03:25:02 INFO mapred.JobClient:     HDFS: Number of read operations=10
14/01/26 03:25:02 INFO mapred.JobClient:     HDFS: Number of large read operations=0
14/01/26 03:25:02 INFO mapred.JobClient:     HDFS: Number of write operations=4
14/01/26 03:25:02 INFO mapred.JobClient:   Job Counters 
14/01/26 03:25:02 INFO mapred.JobClient:     Launched map tasks=2
14/01/26 03:25:02 INFO mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=26203
14/01/26 03:25:02 INFO mapred.JobClient:     Total time spent by all reduces in occupied slots (ms)=0
14/01/26 03:25:02 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/01/26 03:25:02 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=014/01/26 03:25:02 INFO mapred.JobClient:   Map-Reduce Framework
14/01/26 03:25:02 INFO mapred.JobClient:     Map input records=2
14/01/26 03:25:02 INFO mapred.JobClient:     Map output records=0
14/01/26 03:25:02 INFO mapred.JobClient:     Input split bytes=88
14/01/26 03:25:02 INFO mapred.JobClient:     Spilled Records=0
14/01/26 03:25:02 INFO mapred.JobClient:     CPU time spent (ms)=2690
14/01/26 03:25:02 INFO mapred.JobClient:     Physical memory (bytes) snapshot=269602816
14/01/26 03:25:02 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=3734609920
14/01/26 03:25:02 INFO mapred.JobClient:     Total committed heap usage (bytes)=253100032
14/01/26 03:25:02 INFO mapred.JobClient:   Giraph Stats
14/01/26 03:25:02 INFO mapred.JobClient:     Aggregate edges=11
14/01/26 03:25:02 INFO mapred.JobClient:     Aggregate finished vertices=0
14/01/26 03:25:02 INFO mapred.JobClient:     Aggregate vertices=9
14/01/26 03:25:02 INFO mapred.JobClient:     Current master task partition=0
14/01/26 03:25:02 INFO mapred.JobClient:     Current workers=1
14/01/26 03:25:02 INFO mapred.JobClient:     Last checkpointed superstep=0
14/01/26 03:25:02 INFO mapred.JobClient:     Sent messages=0
14/01/26 03:25:02 INFO mapred.JobClient:     Superstep=14
14/01/26 03:25:02 INFO mapred.JobClient:   Giraph Timers
14/01/26 03:25:02 INFO mapred.JobClient:     Setup (milliseconds)=1167
14/01/26 03:25:02 INFO mapred.JobClient:     Shutdown (milliseconds)=232
14/01/26 03:25:02 INFO mapred.JobClient:     Total (milliseconds)=2741giraph exit status: 0 time (sec): 27.4048581123Found 1 items
-rw-r--r--   1 vagrant supergroup         18 2014-01-26 03:24 /tmp/louvain-giraph-example/output/_COMPLETE
giraph times 27.4048581123
total giraph time: 27.4048581123
mapreduce times 
total mapreduce time 0
Total time: 27.4048581123

In [3]:

    

hadoop fs -get /tmp/louvain-giraph-example/output .


    

In [4]:

    

cd output


    

    




/home/vagrant/distributed-louvain-modularity/example/output

In [5]:

    

ls -la


    

    




total 20
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 ./
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 ../
-rwxr-xr-x 1 vagrant vagrant   18 Jan 26 03:25 _COMPLETE*
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 giraph_1/
-rwxr-xr-x 1 vagrant vagrant   18 Jan 26 03:25 _q_1*

In [6]:

    

cat _COMPLETE


    

    




0.4134948096885813

In [7]:

    

cd giraph_1/


    

    




/home/vagrant/distributed-louvain-modularity/example/output/giraph_1

In [8]:

    

ls -la


    

    




total 16
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 ./
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 ../
drwxrwxr-x 3 vagrant vagrant 4096 Jan 26 03:25 _logs/
-rwxr-xr-x 1 vagrant vagrant   98 Jan 26 03:25 part-m-00001*
-rwxr-xr-x 1 vagrant vagrant    0 Jan 26 03:25 _SUCCESS*

In [9]:

    

cat part-m-00001


    

    




4	3	0	3:4
2	3	0	3:4
9	8	0	8:3
5	3	0	3:4
6	8	0	3:1,8:3
3	3	0	3:4
7	8	0	8:3
1	3	0	3:4,8:1
8	8	0	8:3

In [10]:

    

cat ../../small.tsv


    

    




1	0	2:1,3:1,4:1,5:1,6:1
2	0	1:1,3:1,4:1,5:1
3	0	1:1,2:1,4:1,5:1
4	0	1:1,2:1,3:1,5:1
5	0	1:1,2:1,3:1,4:1
6	0	1:1,7:1,8:1,9:1
7	0	6:1,8:1,9:1
8	0	6:1,7:1,9:1
9	0	6:1,7:1,8:1

In [ ]: